Apparatus for processing an audio signal and method thereof

ABSTRACT

An apparatus for processing an audio signal and method thereof are disclosed. The present invention includes receiving a downmix signal and side information; extracting control restriction information from the side information; receiving control information for controlling gain or panning at least one object signal; generating at least one of first multi-channel information and first downmix processing information based on the control information and object information, without using the control restriction information; and, generating an output signal by applying the at least one of the first multichannel information and the first downmix processing information to the downmix signal, wherein the control restriction information relates to a parameter indicating limiting degree of the control information.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a U.S. National Phase Application under 35 U.S.C.§371 of International Application PCT/KR2011/000075, filed on Jan. 6,2011, which claims the benefit of U.S. Provisional Application No.61/292,811, filed on Jan. 6, 2010, U.S. Provisional Application No.61/296,873, filed on Jan. 20, 2010 and U.S. Provisional Application No.61/297,275, filed on Jan. 21, 2010, the entire contents of eachapplication is hereby incorporated by reference in its entirety.

TECHNICAL FIELD

The present invention relates to an apparatus for processing an audiosignal and method thereof. Although the present invention is suitablefor a wide scope of applications, it is particularly suitable forencoding or decoding an audio signal.

BACKGROUND ART

Generally, in the process for downmixing a plurality of objects into amono or stereo signal, parameters are extracted from the object signals,respectively. These parameters are usable for a decoder. And, a panningand gain of each of the objects are controllable via a selection made bya user.

DISCLOSURE OF THE INVENTION Technical Problem

However, in order to control each object signal, each source included ina downmix needs to be appropriately positioned or panned.

Moreover, an object parameter needs to be flexible converted to amultichannel parameter for upmixing to provide a downward compatibilityby a channel-oriented decoding scheme.

Technical Solution

Accordingly, the present invention is directed to an apparatus forprocessing an audio signal and method thereof that substantially obviateone or more of the problems due to limitations and disadvantages of therelated art.

An object of the present invention is to provide an apparatus forprocessing an audio signal and method thereof, by which a gain orpanning of an object is controlled to output a mono signal a stereosignal or a multichannel signal.

Another object of the present invention is to provide an apparatus forprocessing an audio signal and method thereof, by which a distortiongenerated from suppressing or boosting a gain or panning of a specificobject excessively can be controlled when a user adjusts a gain orpanning of an object.

Another object of the present invention is to provide an apparatus forprocessing an audio signal and method thereof, by which a distortionattributed to an excessive rendering in excessively suppressing aspecific object in a karaoke mode, a solo mode or the like can becontrolled in a manner different from that controlling a distortion of aregular object.

Another object of the present invention is to provide an apparatus forprocessing an audio signal and method thereof, by which a modifiedrendering result can be outputted via an interface to be recognized by auser in modifying a control to prevent a distortion due to a user'scontrol.

A further object of the present invention is to provide an apparatus forprocessing an audio signal and method thereof, by which a control forpreventing a distortion can be disabled in accordance with a user'sintention.

Advantageous Effects

Accordingly, the present invention provides the following effects oradvantages.

First of all, in controlling a gain and panning of an object, thepresent invention restricts excessive rendering, thereby enhancing asound quality.

Secondly, when an object signal needs to be excessively controlled incase of a karaoke mode or a solo mode, the present invention is able toefficiently perform a restriction on a control using a separate controlrestriction information.

Thirdly, in case of putting a restriction on a control inputted by auser, the present invention output the restricted control via aninterface, thereby performing the control more efficiently by informinga user that the user-specific control is modified or changed.

Fourthly, in case that a user attempts to apply 100% of a user's controlirrespective of a presence or non-presence of a degradation of a soundquality, the present invention turns of a control for distortionprevention and skips a control restriction information, thereby enablingthe user to perform a user-specific rendering.

DESCRIPTION OF DRAWINGS

The accompanying drawings, which are included to provide a furtherunderstanding of the invention and are incorporated in and constitute apart of this specification, illustrate embodiments of the invention andtogether with the description serve to explain the principles of theinvention.

In the drawings:

FIG. 1 is a block diagram of a configuration of an audio signalprocessing apparatus according to an embodiment of the presentinvention;

FIG. 2 is a block diagram of a configuration of a decoder in an audiosignal processing apparatus according to a first embodiment of thepresent invention;

FIG. 3 is a detailed block diagram of a configuration of an extractingunit 220 shown in FIG. 2 according to an embodiment of the presentinvention;

FIG. 4 is a detailed block diagram of a configuration of a controlrestricting unit 230 shown in FIG. 2 according to a first embodiment230A;

FIG. 5 is a detailed block diagram of a configuration of a controlrestricting unit 230 shown in FIG. 2 according to a second embodiment230B;

FIG. 6 is a diagram for one example of screen configuration of a userinterface 250 shown in FIG. 2;

FIG. 7 is a diagram for one example of a modified control information ina screen displayed on a user interface 250 shown in FIG. 2;

FIG. 8 is a block diagram of a configuration of a decoder in an audiosignal processing apparatus according to a second embodiment of thepresent invention;

FIG. 9 is a detailed block diagram of a configuration of a controlrestricting unit 330 shown in FIG. 8 according to a first embodiment330A;

FIG. 10 is a detailed block diagram of a configuration of a controlrestricting unit 330 shown in FIG. 8 according to a second embodiment330B;

FIG. 11 is a detailed block diagram of a configuration of a controlrestricting unit 330 shown in FIG. 8 according to a third embodiment330B;

FIG. 12 is a diagram for one example of screen configuration for aninput of a control restriction mode off command in a screen displayed ona user interface 340 shown in FIG. 8;

FIG. 13 is a diagram for one example of a user input screen and amodified control information displayed screen in a screen displayed on auser interface 340 shown in FIG. 8;

FIG. 14 is a schematic block diagram of a product in which an audiosignal processing apparatus according to one embodiment of the presentinvention is implemented; and

FIG. 15 is a diagram for explaining relations between products in whichan audio signal processing apparatus according to one embodiment of thepresent invention is implemented.

BEST MODE

Additional features and advantages of the invention will be set forth inthe description which follows, and in part will be apparent from thedescription, or may be learned by practice of the invention. Theobjectives and other advantages of the invention will be realized andattained by the structure particularly pointed out in the writtendescription and claims thereof as well as the appended drawings.

To achieve these and other advantages and in accordance with the purposeof the present invention, as embodied and broadly described, a methodfor processing an audio signal, comprising: receiving a downmix signaland side information; extracting extension type identifier indicatingwhether extension area includes a residual signal from the sideinformation; when the extension type identifier indicates that theextension area includes the residual signal, extracting controlrestriction information for residual using mode from the sideinformation; receiving control information for controlling gain orpanning of at least one object signal; estimating modified controlinformation based on the control information and the control restrictioninformation; obtaining at least one of enhanced object signal and one ormore regular object signals from the downmix signal using the residualsignal; and, generating an output signal using the modified controlinformation and at least one of enhanced object signal and one or moreregular object signal, wherein the control restriction information forresidual using mode relates to a parameter indicating limiting degree ofthe control information in case of the residual using mode is provided.

According to the present invention, the generating of the output signalcomprises: generating downmix processing information based on themodified control information and object information; and, generating theoutput signal by applying the modified control information to at leastone of the enhanced object signal and one or more regular objectsignals.

According to the present invention, a number of channels of the outputsignal is equal to a number of channels of the downmix signal.

According to the present invention, the at least one object signalincludes the enhanced object signal and one or more regular objectsignals.

According to the present invention, the control information comprisesone parameter for controlling gain of the entire one or more regularobject signals.

According to the present invention, when the extension type identifierindicates that the extension area includes the residual signal, thecontrol information includes mode selecting information indicating oneof karaoke mode and solo mode.

To further achieve these and other advantages and in accordance with thepurpose of the present invention, an apparatus for processing an audiosignal, comprising: a demultiplexer receiving a downmix signal and sideinformation; an extracting unit extracting extension type identifierindicating whether extension area includes a residual signal from theside information, and when the extension type identifier indicates thatthe extension area includes the residual signal, extracting controlrestriction information for residual using mode from the sideinformation; a control restricting unit receiving control informationfor controlling gain or panning of at least one object signal, andestimating modified control information based on the control informationand the control restriction information; a residual processing unitobtaining at least one of enhanced object signal and one or more regularobject signals from the downmix signal using the residual signal; and,an object decoder generating an output signal using the modified controlinformation and at least one of enhanced object signal and one or moreregular object signal, wherein the control restriction information forresidual using mode relates to a parameter indicating limiting degree ofthe control information in case of the residual using mode is provided.

According to the present invention, the object decoder comprises: aninformation generating unit generating downmix processing informationbased on the modified control information and object information; and, adownmix processing unit generating the output signal by applying themodified control information to at least one of the enhanced objectsignal and one or more regular object signals.

According to the present invention, a number of channels of the outputsignal is equal to a number of channels of the downmix signal.

According to the present invention, the at least one object signalincludes the enhanced object signal and one or more regular objectsignals.

According to the present invention, the control information comprisesone parameter for controlling gain of the entire one or more regularobject signals.

According to the present invention, when the extension type identifierindicates that the extension area includes the residual signal, thecontrol information includes mode selecting information indicating oneof karaoke mode and solo mode.

To further achieve these and other advantages and in accordance with thepurpose of the present invention, a method for processing an audiosignal, comprising: receiving a downmix signal and side information;extracting control restriction information from the side information;receiving control information for controlling gain or panning at leastone object signal; generating at least one of first multi-channelinformation and first downmix processing information based on thecontrol information and object information, without using the controlrestriction information; and, generating an output signal by applyingthe at least one of the first multichannel information and the firstdownmix processing information to the downmix signal, wherein thecontrol restriction information relates to a parameter indicatinglimiting degree of the control information is provided.

According to the present invention, the method further comprisesextracting off-enable information from the side information, wherein theoff-enable information indicates whether the control restrictioninformation must be used or may be used selectively.

According to the present invention, when the off-enable informationindicates that the control restriction information may be usedselectively, the first multi-channel information and the first downmixprocessing information are generated, without using the controlrestriction information.

According to the present invention, when the off-enable informationindicates that the control restriction information must be used,estimating modified control information based on the control informationand the control restriction information.

According to the present invention, the method further comprisesgenerating at least one of second multichannel information and seconddownmix processing information based on the modified controlinformation, object information and the control restriction information.

According to the present invention, when the output signal correspondsto multichannel signal, the second multichannel information isgenerated, when the output signal corresponds to mono signal or stereosignal, the second multichannel information is not generated.

According to the present invention, when the output signal correspondsto multichannel signal, the first multichannel information is generated.

when the output signal corresponds to mono signal or stereo signal, thefirst multichannel information is not generated.

To further achieve these and other advantages and in accordance with thepurpose of the present invention, an apparatus for processing an audiosignal, comprising: a demultiplexer receiving a downmix signal and sideinformation; an extracting unit extracting control restrictioninformation from the side information; a control restricting unitreceiving control information for controlling gain or panning at leastone object signal; an information generating unit generating at leastone of first multi-channel information and first downmix processinginformation based on the control information and object information,without using the control restriction information; and, an outputchannel generating unit generating an output signal by applying the atleast one of the first multichannel information and the first downmixprocessing information to the downmix signal, wherein the controlrestriction information relates to a parameter indicating limitingdegree of the control information.

According to the present invention, the extracting unit further extractsoff-enable information from the side information, wherein the off-enableinformation indicates whether the control restriction information mustbe used or may be used selectively.

According to the present invention, when the off-enable informationindicates that the control restriction information may be usedselectively, the first multi-channel information and the first downmixprocessing information are generated, without using the controlrestriction information.

According to the present invention, when the off-enable informationindicates that the control restriction information must be used, thecontrol restricting unit estimates modified control information based onthe control information and the control restriction information.

According to the present invention, the information generating unitgenerates at least one of second multichannel information and seconddownmix processing information based on the modified controlinformation, object information and the control restriction information.

According to the present invention, when the output signal correspondsto multichannel signal, the second multichannel information isgenerated, when the output signal corresponds to mono signal or stereosignal, the second multichannel information is not generated.

According to the present invention, when the output signal correspondsto multichannel signal, the first multichannel information is generated,when the output signal corresponds to mono signal or stereo signal, thefirst multichannel information is not generated.

It is to be understood that both the foregoing general description andthe following detailed description are exemplary and explanatory and areintended to provide further explanation of the invention as claimed.

MODE FOR INVENTION

Reference will now be made in detail to the preferred embodiments of thepresent invention, examples of which are illustrated in the accompanyingdrawings. First of all, terminologies or words used in thisspecification and claims are not construed as limited to the general ordictionary meanings and should be construed as the meanings and conceptsmatching the technical idea of the present invention based on theprinciple that an inventor is able to appropriately define the conceptsof the terminologies to describe the inventor's invention in best way.The embodiment disclosed in this disclosure and configurations shown inthe accompanying drawings are just one preferred embodiment and do notrepresent all technical idea of the present invention. Therefore, it isunderstood that the present invention covers the modifications andvariations of this invention provided they come within the scope of theappended claims and their equivalents at the timing point of filing thisapplication.

According to the present invention, terminologies not disclosed in thisspecification can be construed as the following meanings and conceptsmatching the technical idea of the present invention. Specifically,‘coding’ can be construed as ‘encoding’ or ‘decoding’ selectively and‘information’ in this disclosure is the terminology that generallyincludes values, parameters, coefficients, elements and the like and itsmeaning can be construed as different occasionally, by which the presentinvention is non-limited.

FIG. 1 is a block diagram of a configuration of an audio signalprocessing apparatus according to an embodiment of the presentinvention.

Referring to FIG. 1, an encoder 100 includes an object encoder 110 and aside information generating unit 120 and is able to further include amultiplexer 130.

The object encoder 110 generates a downmix signal DMX of a mono orstereo channel by downmixing at least one or more object signals. Inthis case, the at least one or more object signals include at least oneenhanced object signal (EAO) as well as at least one regular objectsignal. For instance, an enhanced object corresponds to an accompaniedmusic in a song and a regular object can correspond to a vocal. In caseof a karaoke mode, the enhanced object remains while the regular objectis suppressed. In case of a solo mode, the regular object remains whilethe enhanced object is suppressed.

The side information generating unit 120 generates a side information onthe at least one object signal in a following manner. First of all, inthe process for downmixing the at least one object signal into a downmixsignal DMX, the side information generating unit 120 generates objectinformation. In this case, the object information (OI) is theinformation on objects included in the downmix signal and is also theinformation necessary to generate a plurality of object signals from thedownmix signal (DMX). In particular, the object information can includeobject level information, object correlation information and the like,by which the present invention is non-limited. Moreover, downmix gaininformation (DMG: DownMix Gain) and downmix channel level difference(DCLD) can be further included in the object information. In this case,the downmix gain information (DMG) indicates a gain applied to eachobject before the corresponding object is downmixed. And, the downmixchannel level difference (DCLD) indicates a ratio of each object appliedto a left channel to the corresponding object applied to a right channelif a downmix signal is a stereo signal.

And, the side information generating unit 120 generates an extensiontype identifier, a residual signal, control restriction information of anormal mode, control restriction information of a residual using mode,and off-enable information. In this case, the control restriction mode(of the normal mode or the residual using mode) corresponds to theinformation for restricting control information determined by a userinput for sound quality enhancement. Such information as the extensiontype identifier and the like shall be described in detail together witha decoding device later.

The multiplexer 130 generates at least one bitstream by multiplexing theside information including the object information, the extension typeidentifier and the like with the downmix (DMX).

FIG. 2 is a block diagram of a configuration of a decoder in an audiosignal processing apparatus according to a first embodiment of thepresent invention.

Referring to FIG. 2, a decoder 200 according to a first embodiment ofthe present invention includes an extracting unit 220 and a controlrestricting unit 230 and is able to further include a demultiplexer 210,a residual processing unit 240, a user interface 250, an informationgenerating unit 260 and a downmix processing unit 270. Optionally, thedecoder 200 is able to further include a multichannel generating unit(not shown in the drawing). Moreover, both of the downmix processingunit 270 and the information generating unit 260 can be named as anobject decoder (not shown in the drawing).

The demultiplexer 210 extracts a downmix signal DMX and side informationfrom at least one received bitstream. As mentioned in the foregoingdescription of the encoder, the downmix signal is a signal generatedfrom downmixing at least one or more object signals and can include amono or stereo signal. In this case, the at least one object signal caninclude an enhanced object as well as a regular object.

The extracting unit 220 extracts object information from the sideinformation. If a residual signal exists in the side information, theextracting unit 220 extracts the residual signal. Moreover, theextracting unit 220 is able to extract an extension type identifier,control restriction information for a residual using mode, of whichdetails shall be explained with reference to FIG. 3 later. The residualsignal is delivered to the residual processing unit 240. The objectinformation is delivered to the information generating unit 260 or canbe delivered to the control restricting unit 230 if necessary.

The control restricting unit 230 receives control information from theuser interface 250 and the like, modifies the received control signal,and then outputs the modified control information. In this case, thecontrol information is provided to control a gain or panning of anobject for at least one object included in the downmix signal.Meanwhile, if the control information has a matrix type, it can berepresented as the following formula.

$\begin{matrix}{M_{ren} = \begin{bmatrix}m_{0,{Lf}} & m_{1,{Lf}} & m_{2,{Lf}} & \ldots & m_{{N - 1},{Lf}} \\m_{0,{Rf}} & m_{1,{Rf}} & m_{2,{Rf}} & \ldots & m_{{N - 1},{Rf}} \\m_{0,C} & m_{1,C} & m_{2,C} & \ldots & m_{{N - 1},C} \\m_{0,{Lfe}} & m_{1,{Lfe}} & m_{2,{Lfe}} & \ldots & m_{{N - 1},{Lfe}} \\m_{0,{Ls}} & m_{1,{Ls}} & m_{2,{Ls}} & \ldots & m_{{N - 1},{Ls}} \\m_{0,{Rs}} & m_{1,{Rs}} & m_{2,{Rs}} & \ldots & m_{{N - 1},{Rs}}\end{bmatrix}} & \lbrack {{Formula}\mspace{14mu} 1} \rbrack\end{matrix}$

In Formula 1, N indicates the total number of objects. And, Lf, Rf, C,Lfe, Ls and Rs indicate a multichannel (i.e., left front channel, rightfront channel, center channel, low frequency channel, left surroundchannel and right surround channel).

If the enhanced object is included in the downmix signal and the sideinformation is included in the residual signal, the control informationcan include mode selection information for the residual using mode(e.g., karaoke mode, solo mode, etc.).

Meanwhile, the control restricting unit 230 is able to use an objectsignal (particularly, downmix gain information (DMG), controlrestriction information and/or the like to modify or transform thecontrol information, which shall be described in detail with referenceto FIG. 4 and FIG. 5 later. The control restricting unit 230 deliversthe modified control information to the information generating unit 260or to the user interface 250. In case that the modified controlinformation includes a control on the enhanced object, it can bedelivered to the residual processing unit 240 as well.

The user interface 250 receives a command for a user-specific objectcontrol. If a residual is included in a downmix signal and a residualusing mode (e.g., karaoke mode, solo mode, etc.) is available, it isable to provide a separate screen for the control on the enhanced objectin the residual using mode, which shall be described with reference toFIG. 6 later. Besides, it is able to output the modified controlinformation received from the control restricting unit 230, which shallbe described with reference to FIG. 7 later.

In case of receiving the residual signal from the extracting unit 220,the residual processing unit 240 extracts the enhanced object (EAO)and/or the at least one regular object from the downmix signal DMX usingthe received residual signal. In this case, the residual signal mayinclude the signal generated in the course of downmixing the enhancedobject and the regular objects in the encoder or can include the signalnecessary to extract the enhanced object or the at least one regularobject from the downmix signal DMX. Meanwhile, in the process forextracting the enhanced object (EAO), object information including anobject level difference and the like can be further used as well as theresidual signal.

Afterwards, if the control on the enhanced object (EAO) is included inthe control information, the residual processing unit 240 performs thecontrol on the enhanced object based on the modified controlinformation. Subsequently, the residual processing unit 240 delivers theat least one regular object and the enhanced object (EAO) to the downmixprocessing unit 270.

The information generating unit 260 generates downmix processinginformation using the object information received from the extractingunit 220 and the modified control information received from the controlrestricting unit 230. In this case, the downmix processing informationis the information applied to the enhanced object and/or the regularobject to generate a processed downmix signal. If a multichannel outputis necessary, the information generating unit 260 is able to furthergenerate multichannel information using the object information and themodified control information. In this case, the multichannel informationis the information used to generate a multichannel by upmixing thedownmix signal and is able to include such a spatial parameter as achannel level difference (CLD) and the like. In particular, themultichannel information includes frame information (Frameinfo ( )), OTTinformation (OttData ( )), and the like. The frame information(Frameinfo ( )) can include the number of parameter sets and frameinformation indicating the parameter set is applied to which time slot.And, the OTT information can include such a parameter necessary for anOTT (one-to-two) box as a channel level difference (CLD), a channelcorrelation information (ICC) and the like.

Yet, if there exists a condition that the number of channels of adownmix signal should be equal to that of an output signal amongconditions of the residual using mode, the information generating unit260 does not generate the multichannel information but does the downmixprocessing information only. On this condition, the control informationreceived by the control restricting unit 230 can have a specializedconfiguration by schematizing Formula 1 into a stereo or mono channelinstead of the multichannel.

The downmix processing unit 270 outputs a processed downmix signal byprocessing the enhanced object and/or the regular object using thedownmix processing information. If the above-mentioned condition exists,the downmix processing unit 270 outputs the processed downmix signal(i.e., the processed enhanced object and/or the processes regularobject) as an output signal of a time domain.

FIG. 3 is a detailed block diagram of a configuration of the extractingunit 220 shown in FIG. 2 according to an embodiment of the presentinvention.

Referring to FIG. 3, the extracting unit 220 can include an identifierextracting part 222, a restriction information extracting part 224 and aresidual extracting part 226.

The identifier extracting part 222 extracts an extension type identifierfrom side information. In this case, the extension type identifierincludes an identifier indicting whether an extension area of the sideinformation includes a residual signal And, the extracted extension typeidentifier is delivered to the restriction information extracting part224.

If the extension type identifier indicates that the extension areaincludes the residual signal, the restriction information extractingpart 224 extracts control restriction information for the residual usingmode from the side information.

In this case, the control restriction information is the informationputting restriction on the control information inputted by a userthrough the user interface 250 in order to prevent a sound quality frombeing distorted or degraded. Since at least one or more objectsdownmixed into a downmix signal have correlation, if a specific objectis excessively suppressed or boosted, it may cause a problem that asound quality can be degraded. If a user attempt to perform an extremerendering, the control restriction information plays a role in puttingrestriction enough not to considerably distort the sound quality. Thecontrol restriction information is associated with a parameter thatindicates a limiting degree for the control information.

Meanwhile, the control restriction information for the residual usingmode means the control restriction information suitable for the residualusing mode (e.g., karaoke mode, solo mode, etc.). For instance, thekaraoke or solo mode is the mode for performing an extreme rendering ofsuppressing all objects except a background music or suppressing allobjects except a vocal. Therefore, if the control restrictioninformation of a normal mode is applied as it is, the original purposeof the karaoke or solo mode may not be encouraged. For instance, a vocalcomponent remains in the karaoke mode or a background music componentmay be mixed in the solo mode. Hence, if a residual signal is receivedfor an extreme rendering, control restriction information suitable for aresidual using mode is separately received. The control restrictioninformation of the residual using mode is delivered to the formercontrol restricting unit 230 described with reference to FIG. 2.

If the extension type identifier indicates that the extension areaincludes the residual signal, the residual extracting part 226 extractsthe residual signal from the side information and then delivers theresidual signal to the former residual processing unit 240 describedwith reference to FIG. 2.

FIG. 4 is a detailed block diagram of a configuration of a controlrestricting unit 230 shown in FIG. 2 according to a first embodiment230A. And, FIG. 5 is a detailed block diagram of a configuration of acontrol restricting unit 230 shown in FIG. 2 according to a secondembodiment 230B. In particular, in case of a residual using mode, thecontrol restricting unit 230A according to a first embodiment modifiescontrol information using separate control restriction information foran enhanced object. In case of a residual using mode, the controlrestricting unit 230B according to a second embodiment modifies controlinformation by restricting a control on a regular object except anenhanced object.

Referring to FIG. 4, the control restricting unit 230A includes a modedeciding part 232A and a control information modifying part 234A.

The mode deciding part 232A is the part configured to decide a presenceor non-presence of a residual using mode (e.g., karaoke mode and/or solomode). The presence or non-presence of the residual using mode can bedecided in a manner of interpreting data inputted by a user to control again and/or panning or can be decided based on whether a separate inputfor selecting a karaoke mode or a solo mode is received. In the lattercase, if control information is inputted via a separate karaoke tapshown in FIG. 6 or FIG. 7, the inputted control information can beregarded as an input for selecting a karaoke mode or a solo mode.

If an enhanced object is included in a downmix, the mode deciding part232A is able to determine which one of a plurality of objects includedin the downmix is the enhanced object as well as determines the presenceor non-presence of the residual using mode.

If the presence of the residual using mode is decided, the controlinformation modifying part 234A modifies the control information usingthe control restriction information for the residual using mode and thenoutputs the modified control information. The control restrictioninformation for the residual using mode is associated with a parameterindicating a limiting degree of the control information in case of theresidual using mode. For instance, since an extreme rendering isgenerally allowed for the residual using mode, a value of the parameterindicating the limiting degree can be smaller than a parameter value ofa normal mode.

On the contrary, if the non-presence of the residual using mode isdecided, the control information modifying part 234A is able to modifythe control information using the control restriction information forthe normal mode.

Meanwhile, if a condition that regular objects are controllable as onein the residual using mode instead of being controlled individually isadded, the control information received by the control informationmodifying part 234A can include the information interoperable amongregular objects except an enhanced object. For instance, via such ascreen as shown in FIG. 6 (B), the interoperable control information canbe generated based on the information inputted via coupled panning knobsor coupled gain control sliders.

Thus, the control restricting unit 230A according to the firstembodiment receives the control restriction information for the residualusing mode. In case of the residual using mode, the control restrictingunit 230A modifies control information on the objects including theenhanced object using the received control restriction information.

Referring to FIG. 5, a control restricting unit 230B according to asecond embodiment includes a mode deciding part 232B and a controlinformation modifying part 2348. Since the mode deciding part 232Bperforms a function almost similar to that of the former mode decidingpart 232A of the first embodiment, its description shall be omitted.

Unlike the former control restricting unit 230B according to the firstembodiment, in case of a residual using mode, the control informationmodifying part 234B does not modify control information for an enhancedobject but modifies control information on a regular object. In thiscase, control restriction information used for the modification can bereceived or self-generated. And, the control restriction information maybe free from the condition that regular objects are controlled at atime.

The control information on the enhanced object is not modified but thecontrol information on the regular object is modified. Thus, one exampleof the modified control information (M_(ren,limited)) is represented asFormula 2 and Formula 3.

$\begin{matrix}{M_{{ren},{limited}} = {\quad\begin{bmatrix}m_{0,{Lf}} & m_{1,{Lf}} & {\hat{m}}_{2,{Lf}} & \ldots & {\hat{m}}_{{N - 1},{Lf}} \\m_{0,{Rf}} & m_{1,{Rf}} & {\hat{m}}_{2,{Rf}} & \ldots & {\hat{m}}_{{N - 1},{Rf}} \\m_{0,C} & m_{1,C} & {\hat{m}}_{2,C} & \ldots & {\hat{m}}_{{N - 1},C} \\m_{0,{Lfe}} & m_{1,{Lfe}} & {\hat{m}}_{2,{Lfe}} & \ldots & {\hat{m}}_{{N - 1},{Lfe}} \\m_{0,{Ls}} & m_{1,{Ls}} & {\hat{m}}_{2,{Ls}} & \ldots & {\hat{m}}_{{N - 1},{Ls}} \\m_{0,{Rs}} & m_{1,{Rs}} & {\hat{m}}_{2,{Rs}} & \ldots & {\hat{m}}_{{N - 1},{Rs}}\end{bmatrix}}} & \lbrack {{Formula}\mspace{14mu} 2} \rbrack \\\begin{matrix}{\mspace{155mu}{unmodified}} & {\mspace{56mu}{modified}}\end{matrix} & \;\end{matrix}$

In Formula 2, N indicates the total number of objects. Lf, Rf, C, Lfe,Ls and Rs indicate a multichannel (i.e., left front channel, right frontchannel, center channel, low frequency channel, left surround channeland right surround channel). M_(ren,limited) indicates a modifiedcontrol information.

In Formula 2, enhanced objects include a first object and a secondobject (j=0, 1).M _(ren,limited) =I _(EAO) M _(ren)+(I−I _(EAO))M _(ren,limited) _(—)_(temp)  [Formula 3]

In Formula 3, M_(ren) indicates a received control information [cf.Formula 1].

When the total object number is N, an index of an enhanced object is j,and N×N indicates a diagonal matrix, I_(EAO) indicates a matrix havingj^(th) diagonal element set to 1 and the rest of elements set to 0,i.e., an element that passes a column corresponding to the enhancedobject.

M_(ren,limited) _(—) _(temp) indicates modified control information,which is modified using control restriction information of a normalmode, on regular objects.

Meanwhile, the control restriction information of the normal mode, whichis used to modify the control information on the regular object, isreceived via a bitstream or can be generated via downmix gaininformation (DMG) in object information. As mentioned in the foregoingdescription, the downmix gain information includes a gain indicating anextent of each object included in a downmix (DMX). Hence, when a usercontrols a gain or panning of an object based on the downmix gaininformation, it is able to determine what degree of limitation will beput (i.e., a limiting degree).

In the foregoing description, the decoder according to the firstembodiment is described with reference to FIG. 2, the detailedconfiguration of the extracting unit 220 is examined with reference toFIG. 3, and the two kinds of embodiments of the control restricting unit230 are explained with reference to FIG. 4 and FIG. 5.

FIG. 6 is a diagram for one example of screen configuration of a userinterface 250 shown in FIG. 2. and, FIG. 7 is a diagram for one exampleof a modified control information in a screen displayed on a userinterface 250 shown in FIG. 2.

Referring to FIG. 6, in case of a residual using mode (e.g., karaokemode, solo mode) in FIG. 6 (A) or FIG. 6 (B), a separate input window(i.e., a karaoke tap) can be provided. As mentioned in the foregoingdescription, control information inputted via the input window of theresidual using mode can be regarded as an input for selecting theresidual using mode (e.g., karaoke mode or solo mode).

Referring to FIG. 6 (A), total 4 objects obj1 to obj4 exist in aseparate input window for a residual using mode. And, an enhanced objectobj1 (EAO) is included in the 4 objects as well as regular objects obj2to obj4. And, gain control sliders capable of adjusting gains of theobjects respectively and panning knobs capable of adjusting pannings ofthe objects respectively exist in the separate input window.

Meanwhile, as mentioned in the foregoing description of the controlrestricting unit 230A of the first embodiment, there may be a followingcondition. First of all, if a control restriction is performed despitethe existence of an enhanced object, it is impossible to control each ofregular objects. Hence, all of the objects should be controlled at atime. And, FIG. 6 (B) shows one example of a screen if such a conditionexists. Although a gain and/or panning of an enhanced object isindividually possible, since regular objects should be collectivelyadjusted at a time, there exist simultaneously-adjustable coupled gaincontrol sliders and simultaneously-adjustable coupled panning knobs. Ifa user manipulates a slider belonging to the coupled gain controlsliders (or a panning knob belonging to the coupled panning knobs), thecoupled gain control sliders or the coupled panning knobs are entirelyadjusted. In this case, it is preferable to display an image showingthat the coupled gain control sliders or the coupled panning knobs arecollectively shifted at a time.

FIG. 7 shows one example of a modified control information in the screendisplayed on the user interface 250 shown in FIG. 2. After controlinformation has been modified, if the modification of the controlinformation is not displayed, a user may attempt to further raise acontrol degree. For instance, when a user performs a control to raise again of a specific object, as the corresponding control is restricted,if an audio is outputted with a gain lower than a user-specific level,the user misunderstands that the control is no sufficient and mayconsider raising the gain higher. Therefore, if the control inputted bya user is modified, it is preferable that the modified controlinformation is outputted via the user interface. Likewise, in case of aresidual using mode, when a control restriction is performed, themodified control information is preferably displayed on the screen.

Referring to FIG. 7, it can be observed that control information, whichis modified based on control restriction information, is displayed as animpossible area. Although a user may be able to raise a gain controlslider up to the impossible area, the modified control information canbe represented as turning back below the impossible area. Alternatively,it is able to set the gain control slider to be raised by a user justbelow the impossible area. A method of displaying or representing themodified control information is non-limited by a specific scheme andscreen.

FIG. 8 is a block diagram of a configuration of a decoder in an audiosignal processing apparatus according to a second embodiment of thepresent invention. FIG. 9 is a detailed block diagram of a configurationof a control restricting unit 330 shown in FIG. 8 according to a firstembodiment 330A. FIG. 10 is a detailed block diagram of a configurationof a control restricting unit 330 shown in FIG. 8 according to a secondembodiment 330B. And, FIG. 11 is a detailed block diagram of aconfiguration of a control restricting unit 330 shown in FIG. 8according to a third embodiment 330B. Unlike the former decoder 200 ofthe first embodiment, a decoder 300 according to a second embodimentincludes an embodiment corresponding to a normal mode rather tan aresidual using mode. Occasionally, the second embodiment may, not modifycontrol information (per frame).

Referring to FIG. 8, a decoder 300 according to a second embodimentincludes an extracting unit 320 and a control restricting unit 330 andis able to further include a demultiplexer 310, a user interface 340, aninformation generating unit 350, a downmix processing unit 360 and amultichannel generating unit 370. Meanwhile, both of the downmixingprocessing unit 360 and the multichannel generating unit 270 can benamed an output channel generating unit.

Like the former demultiplexer 210 of the first embodiment, thedemultiplexer 310 receives a downmix signal and side information from atleast one bitstream.

The extracting unit 320 extracts object information from sideinformation and also extracts control restriction information (for anormal mode). The object information and the control restrictioninformation can refer to the former description with reference to FIG.2.

The extracting unit 320 is able to further extract enable-offinformation from the side information. In this ease, the enable-offinformation is the information indicating whether a use of controlrestriction information for a current frame is mandatory or optional, ifthe control restriction information exists.

The control restricting unit 330 receives the control restrictioninformation from the extracting unit 320 and then determines whether toapply the received control restriction information to the current framebased on the enable-off information and/or a user-off command.Irrespective of a presence or non-presence of a degradation of a soundquality, if a user does not want a specific control to be restricted,although the control restriction information is included in thebitstream, the use of the control restriction information can beskipped.

If the control restricting unit 330 determines to apply the controlrestriction information, the control restricting unit 330 modifies thecontrol information using the control restriction information and thenoutputs the modified control information. On the contrary, if thecontrol restricting unit 330 determines not to apply the controlrestriction information, the control restricting unit 330 does not usethe control restriction information but delivers the received controlinformation to the information generating unit 350 as it is. The controlinformation and the modified control information can refer to thedescription of the former decoder 200 of the first embodiment. Yet, thecontrol information (and the control restriction information) of thesecond embodiment corresponds to control information (and controlrestriction information) suitable for a normal mode rather than controlinformation (and control restriction information) suitable for aresidual using mode.

Meanwhile, a third embodiment of a control restricting unit 330 isprovided to propose a detailed formula in applying control restrictioninformation rather than determining whether to apply control restrictioninformation each frame.

If control information is modified, the user interface 340 is able tooutput the modified control information. FIG. 12 is a diagram for oneexample of screen configuration for an input of a control restrictionmode off command in a screen displayed on the user interface 340 shownin FIG. 8. FIG. 13 is a diagram for one example of a user input screenand a modified control information displayed screen in a screendisplayed on the user interface 340 shown in FIG. 8. FIG. 12 shall bedescribed together with FIG. 9 later.

Referring to FIG. 13, similar to the case shown in FIG. 7, it can beobserved that the modified control information is represented as animpossible area. Even if a user-specific gain exists in an impossiblearea, as shown in FIG. 13 (A), it is able to adjust a gain controlslider to enter the impossible area. In this case, a modified gainaccording to modified control information can be represented as shown inFIG. 13 (B). In particular, since the modified gain can have a maximumor minimum value except the impossible area, the gain control slider isdisplayed as moving to a corresponding maximum or minimum value area.

On the contrary, if a user-specific gain exists in the impossible area,it is able to display the modified control information in a manner ofenabling the gain control slider not to be adjusted into the impossiblearea at all.

Referring now to FIG. 8, the information generating unit 350 receivesthe object information having received from the extracting unit 220. Inthis case, the object information is as good as the former descriptionof the first embodiment 200. Afterwards, in case of receiving thecontrol information from the control restricting unit 230 for a currentframe, the information generating unit 350 generates a first downmixprocessing information (and a first multichannel information). In caseof receiving the modified information from the control restricting unit230 for the current frame instead of receiving the control information,the information generating unit 350 generates a second downmixprocessing information (and a second multichannel information). Asmentioned in the foregoing description of the first embodiment 100, eachof the first downmix processing information and the second downmixprocessing information is the information for processing the downmixsignal by avoiding incrementing the number of channels. As mentioned inthe foregoing description of the first embodiment 100, each of the firstmultichannel information and the second multichannel information is theinformation for generating a multichannel by upmixing the downmix signalor the processed downmix signal.

The information generating unit 350 generates the first downmixprocessing information and the second downmix processing informationonly in a decoding mode, in which a final output signal is a mono orstereo signal. The information generating unit 350 further generates thefirst multichannel signal and the second multichannel signal in atranscoding mode, in which a final output signal is a multichannelsignal. If an input and an output are a mono downmix signal and amultichannel, respectively, the downmix processing information is notgenerated but the multichannel information is generated only.

The downmix processing unit 360 receives a downmix signal DMX, processesthe downmix signal DMX based on the downmix processing information, andthen outputs a processed downmix. In this case, the channel number ofthe downmix signal is equal to that of the processed downmix signal.Since the downmix processing information is used, the downmix processingunit 360 corresponds to an output in which a gain and/or panning of anobject has been adjusted. In particular, in case of a stereo downmix anda multichannel output, the downmix processing unit 360 is able toperform a control of object panning by cross-term. Meanwhile, in case ofa mono downmix and a multichannel output, a process for the downmix DMXbypasses the downmix processing unit 360 and the downmix DMX can bedelivered to the multichannel generating unit 370 without change.Besides, in case of a mono or stereo output, the downmix processing unit360 is able to output the processed downmix signal into a mono or stereochannel in time domain.

If a multichannel output is necessary, the multichannel generating unit370 generates the multichannel output by applying the multichannelinformation (i.e., the first multichannel information or the secondmultichannel information) to the downmix signal DMX or the processeddownmix signal.

In the following description, first to third embodiments 330A to 330C ofthe control restricting unit 330 are explained with reference to FIGS. 9to 11. The first embodiment 330A relates to an embodiment for decidingwhether to apply control restriction information based on an off commandinputted by a user or a device setting information. The secondembodiment 330B relates to an embodiment for deciding whether to applycontrol restriction information by further referring to enable-offinformation extracted from a bitstream in addition to the off command orthe device setting information. And, the third embodiment 330C relatesto an embodiment for proposing a detailed formula in applying controlrestriction information rather than deciding whether to apply thecontrol restriction information each frame.

Referring to FIG. 9, a control restricting unit 330A according to athird embodiment includes an off deciding part 332A and a controlinformation modifying part 334A.

The control restricting unit 330A decides whether to turn off a controlrestriction mode based on a user's of command or a device settinginformation. In this case, the control restriction mode means that amodified control information is generated to restrict controlinformation using a control restriction information. Even if a controlrestriction information exists in a bitstream or a decoder is able togenerate a control restriction information by itself, a user may want acontrol information inputted by himself not to be modified at all. Forthis case, the control restricting unit 330A is able to decide to turnoff the control restriction mode. In particular, in case that an offcommand for turning off a control restriction mode is inputted by a useror a command for tuning off a control restriction mode is stored in adevice setting information, it is able to decide to turn of the controlrestriction mode. One example of a screen for the off command is shownin FIG. 12.

Referring to FIG. 12 (A) and FIG. 12 (B), an input unit (e.g., a radiobutton) for an on command (ON) of a control restriction mode and aninput unit (off command) for an off command (OFF) are displayed on abottom of a screen. A user is then able to input the off command forturning off the control restriction mode for a specific audio signal viathis screen. Moreover, the user is able to change the device settinginformation to turn off the control restriction mode for a non-specifiedaudio signal.

Referring now to FIG. 9, if the off deciding part 332A decides to turnoff the control restriction mode, it outputs the control information asit is. As mentioned in the foregoing description, the controlinformation is delivered to the information generating unit 350 shown inFIG. 5. On the contrary, if the off deciding part 332A decides to turnon the control restriction mode, it delivers the control information tothe control information modifying part 334A. Subsequently, the controlinformation modifying part 334A generates a modified control informationby putting restriction on the control information using the controlrestriction information. And, the modified control information isdelivered to the information generating unit 350 as well. As theconfiguration and role of the control information modifying part 334 canbe similar to those of the former control information modifying part234A or 234B, their details shall be omitted from the followingdescription.

Referring to FIG. 10, like the first embodiment, a control restrictingunit 330B according to a second embodiment includes an off deciding part332B and a control information modifying part 334B. Yet, the secondembodiment differs from the first embodiment in that the off decidingpart 332B further refers to enable-off information in deciding whetherto turn off a control restriction mode. In this case, the enable-offinformation is the information extracted from the side information bythe extracting unit 320. And, the enable-off information indicateswhether a use of control restriction information for a current frame ismandatory or optional. In other words, the mandatory use of the controlrestriction information is related to whether a user is able toarbitrarily skip the control restriction information. Hence, theenable-off information can have the same idea of whether a controlrestriction mode can be turned off or whether the control restrictionode can be selectively turned off.

The enable-off information can be extracted only if the controlrestriction information is received via a bitstream. In this case, theenable-off information indicates whether a use of the controlrestriction information included in the bitstream is mandatory or not.

Therefore, if the enable-off information indicates that the use of thecontrol restriction information (for a current frame) is mandatory, theoff deciding part 332B does not turn off the control restriction modeirrespective of a user's off input. In particular, the off deciding part332B delivers the control information to the control informationmodifying part 334B.

On the contrary, if the enable-of information indicates that the use ofthe control restriction information is not mandatory but optional, theoff deciding part 332B decides whether to turn off the controlrestriction mode according to a user's off command or a device settinginformation. In particular, if the off command or the device settinginformation intends to turn off the control restriction mode, the offdeciding part 332B outputs the control information as it is. If the offcommand or the device setting information intends to turn on the controlrestriction mode, the off deciding part 332B delivers the controlinformation to the control information modifying part 334B.

As the control information modifying part 334B outputs the modifiedcontrol information in the same manner of the former control informationmodifying part 334B described with reference to FIG. 9, its detailsshall be omitted from the following description.

FIG. 11 shows a control restricting unit 330C according to a thirdembodiment. As mentioned in the foregoing description, the thirdembodiment is provided to propose a detailed formula in applying controlrestriction information rather than determining whether to apply controlrestriction information each frame.

A control information modifying part 334C receives downmix gaininformation DMG and control restriction information and then generates amodified control information by restricting control information based onthe downmix gain information DMG and the control restrictioninformation. In doing so, the following formula is usable.M _(ren,lim) ^(l,m)=(1−g _(DCU))M _(ren) ^(l,m) +g _(DCU) D _(DS)^(l)  [Formula 4]

In Formula 4, M_(ren,limited) ^(l,m) indicates a modified controlinformation, M_(ren) ^(l,m) indicates a control information, g_(DCU)indicates a parameter corresponding to a control restriction information(or a parameter indicating a limiting degree), and D_(DS) ^(l) indicatesa downmix matrix defined by a downmix gain information DMG.

Referring to Formula 4, if the parameter g_(DCU) indicating the limitingdegree has a lower value, the modified control information gets closerto an original control information. On the contrary, if the parameterg_(DCU) indicating the limiting degree has a higher value, the modifiedcontrol information gets closer to an original downmix matrix.

According to formula 4, it is able to solve the problem that themodified control information M_(ren,lim) ^(l,m) is generated unsuitablefor a user's intention. When a user inputs control information [1, 3] toboost a second object, if the parameter g_(DCU) is 1, the modifiedcontrol information results in [2.8, 1.4]. If so, it may cause a problemthat the second object is suppressed to the contrary. Yet, if themodified control information defined by Formula 4, it is able to avoidthe above-mentioned problem and can be advantageous in that an operationamount is very small.

Meanwhile, the g_(DCU) can have a value independent per object insteadof being in common with every object. If so, Formula 4 can be modifiedinto a form of applying a different g_(DCU) per column corresponding toeach object.

Meanwhile, in case of the implementation according to Formula 4, ifFormula 4 is applied to a case that a control restriction mode is ‘ON’,it is not classified into a first mode or a second mode. Hence, it isunnecessary to transmit a separate bit to identify a mode.

The audio signal processing apparatus according to the present inventionis available for various products to use. Theses products can be mainlygrouped into a stand alone group and a portable group. A TV, a monitor,a settop box and the like can be included in the stand alone group. And,a PMP, a mobile phone, a navigation system and the like can be includedin the portable group.

FIG. 14 shows relations between products, in which an audio signalprocessing apparatus according to an embodiment of the present inventionis implemented.

Referring to FIG. 14, a wire/wireless communication unit 410 receives abitstream via wire/wireless communication system. In particular, thewire/wireless communication unit 410 can include at least one of a wirecommunication unit 410A, an infrared unit 410B, a Bluetooth unit 410Cand a wireless LAN unit 410D.

A user authenticating unit 420 receives an input of user information andthen performs user authentication. The user authenticating unit 420 caninclude at least one of a fingerprint recognizing unit 420A, an irisrecognizing unit 420B, a face recognizing unit 420C and a voicerecognizing unit 420D. The fingerprint recognizing unit 420A, the irisrecognizing unit 420B, the face recognizing unit 420C and the speechrecognizing unit 420D receive fingerprint information, iris information,face contour information and voice information and then convert theminto user informations, respectively. Whether each of the userinformations matches pre-registered user data is determined to performthe user authentication.

An input unit 430 is an input device enabling a user to input variouskinds of commands and can include at least one of a keypad unit 430A, atouchpad unit 430B and a remote controller unit 430C, by which thepresent invention is non-limited.

A signal coding unit 440 performs encoding or decoding on an audiosignal and/or a video signal, which is received via the wire/wirelesscommunication unit 410, and then outputs an audio signal in time domain.The signal coding unit 440 includes an audio signal processing apparatus445. As mentioned in the foregoing description, the audio signalprocessing apparatus 445 corresponds to the above-described embodiment(i.e., the encoder 100 and/or the decoder 200) of the present invention.Thus, the audio signal processing apparatus 445 and the signal codingunit including the same can be implemented by at least one or moreprocessors.

A control unit 450 receives input signals from input devices andcontrols all processes of the signal decoding unit 440 and an outputunit 460. In particular, the output unit 460 is a component configuredto output an output signal generated by the signal decoding unit 440 andthe like and can include a speaker unit 460A and a display unit 460B. Ifthe output signal is an audio signal, it is outputted to a speaker. Ifthe output signal is a video signal, it is outputted via a display.

FIG. 15 is a diagram for relations of products provided with an audiosignal processing apparatus according to one embodiment of the presentinvention. FIG. 15 shows the relation between a terminal and servercorresponding to the products shown in FIG. 14.

Referring to FIG. 15 (A), it can be observed that a first terminal 400.1and a second terminal 400.2 can exchange data or bitstreamsbi-directionally with each other via the wire/wireless communicationunits. Referring to FIG. 15 (B), it can be observed that a server 500and a first terminal 500.1 can perform wire/wireless communication witheach other.

An audio signal processing method according to the present invention canbe implemented into a computer-executable program and can be stored in acomputer-readable recording medium. And, multimedia data having a datastructure of the present invention can be stored in thecomputer-readable recording medium. The computer-readable media includeall kinds of recording devices in which data readable by a computersystem are stored. The computer-readable media include ROM, RAM, CD-ROM,magnetic tapes, floppy discs, optical data storage devices, and the likefor example and also include carrier-wave type implementations (e.g.,transmission via Internet). And, a bitstream generated by the abovementioned encoding method can be stored in the computer-readablerecording medium or can be transmitted via wire/wireless communicationnetwork.

INDUSTRIAL APPLICABILITY

Accordingly, the present invention is applicable to process and outputan audio signal.

While the present invention has been described and illustrated hereinwith reference to the preferred embodiments thereof, it will be apparentto those skilled in the art that various modifications and variationscan be made therein without departing from the spirit and scope of theinvention. Thus, it is intended that the present invention covers themodifications and variations of this invention that come within thescope of the appended claims and their equivalents.

What is claimed is:
 1. A method of processing an audio signal, themethod comprising: receiving a downmix signal and side information;extracting control restriction information and off-enable informationfrom the side information; receiving control information for controllinggain or panning at least one object signal; generating at least one offirst multi-channel information or first downmix processing informationbased on the off-enable information, the control information and objectinformation, without using the control restriction information; and,generating an output signal by applying the at least one of the firstmultichannel information or the first downmix processing information tothe downmix signal, wherein the control restriction information relatesto a parameter indicating limiting degree of the control information,and wherein the off-enable information indicates whether the controlrestriction information must be used or may be used selectively.
 2. Themethod of claim 1, wherein, based on the off-enable informationindicating that the control restriction information may be usedselectively, the first multi-channel information and the first downmixprocessing information are generated, without using the controlrestriction information.
 3. The method of claim 1, further comprising,based on the off-enable information indicating that the controlrestriction information must be used, estimating modified controlinformation based on the control information and the control restrictioninformation.
 4. The method of claim 3, further comprising: generating atleast one of second multichannel information or second downmixprocessing information based on the modified control information, objectinformation and the control restriction information.
 5. The method ofclaim 4, wherein, based on the output signal corresponding tomultichannel signal, the second multichannel information is generated,and based on the output signal corresponding to mono signal or stereosignal, the second multichannel information is not generated.
 6. Themethod of claim 1, wherein, based on the output signal corresponding tomultichannel signal, the first multichannel information is generated,based on the output signal corresponding to mono signal or stereosignal, the first multichannel information is not generated.
 7. Anapparatus configured to process an audio signal, the apparatuscomprising: a demultiplexer configured to receive a downmix signal andside information; an extracting unit configured to extract controlrestriction information and off-enable information from the sideinformation; a control restricting unit configured to receive controlinformation for controlling gain or panning at least one object signal;an information generating unit configured to generate at least one offirst multi-channel information or first downmix processing informationbased on the off-enable information, the control information and objectinformation, without using the control restriction information; and, anoutput channel generating unit configured to generate an output signalby applying the at least one of the first multichannel information orthe first downmix processing information to the downmix signal, whereinthe control restriction information relates to a parameter indicatinglimiting degree of the control information, and wherein the off-enableinformation indicates whether the control restriction information mustbe used or may be used selectively.
 8. The apparatus of claim 7, whereinthe information generating unit is further configured to generate, basedon the off-enable information indicating that the control restrictioninformation may be used selectively, the first multi-channel informationand the first downmix processing information without using the controlrestriction information.
 9. The apparatus of claim 7, wherein thecontrol restricting unit is further configured to estimate, based on theoff-enable information indicating that the control restrictioninformation must be used, modified control information based on thecontrol information and the control restriction information.
 10. Theapparatus of claim 9, wherein the information generating unit is furtherconfigured to generate at least one of second multichannel informationor second downmix processing information based on the modified controlinformation, object information and the control restriction information.11. The apparatus of claim 10, wherein the information generating unitis further configured to generate, based on the output signalcorresponding to multichannel signal, the second multichannelinformation, and the information generating unit is further configuredto not generate, based on the output signal corresponding to mono signalor stereo signal, the second multichannel information.
 12. The apparatusof claim 7, wherein the information generating unit is furtherconfigured to generate, based on the output signal corresponding tomultichannel signal, the first multichannel information, and theinformation generating unit is further configured to not generate, basedon the output signal corresponding to mono signal or stereo signal, thefirst multichannel information.